Search for: All records

The advent of artificial intelligence and machine learning has led to significant technological and scientific progress, but also to new challenges. Partial differential equations, usually used to model systems in the sciences, have shown to be useful tools in a variety of tasks in the data sciences, be it just as physical models to describe physical data, as more general models to replace or construct artificial neural networks, or as analytical tools to analyse stochastic processes appearing in the training of machine-learning models. This article acts as an introduction of a theme issue covering synergies and intersections of partial differential equations and data science. We briefly review some aspects of these synergies and intersections in this article and end with an editorial foreword to the issue. This article is part of the theme issue ‘Partial differential equations in data science’. 

Abstract In the (special) smoothing spline problem one considers a variational problem with a quadratic data fidelity penalty and Laplacian regularization. Higher order regularity can be obtained via replacing the Laplacian regulariser with a poly-Laplacian regulariser. The methodology is readily adapted to graphs and here we consider graph poly-Laplacian regularization in a fully supervised, non-parametric, noise corrupted, regression problem. In particular, given a dataset$$\{x_i\}_{i=1}^n$$ ${\left\{x_i\right\}}_{i=1}^n$and a set of noisy labels$$\{y_i\}_{i=1}^n\subset \mathbb {R}$$ ${\left\{y_i\right\}}_{i=1}^n\subset R$we let$$u_n{:}\{x_i\}_{i=1}^n\rightarrow \mathbb {R}$$ $u_n:{\left\{x_i\right\}}_{i=1}^n\to R$be the minimizer of an energy which consists of a data fidelity term and an appropriately scaled graph poly-Laplacian term. When$$y_i = g(x_i)+\xi _i$$ $y_i=g\left(x_i\right)+{\xi }_i$, for iid noise$$\xi _i$$ ${\xi }_i$, and using the geometric random graph, we identify (with high probability) the rate of convergence of$$u_n$$ $u_n$togin the large data limit$$n\rightarrow \infty $$ $n\to \infty$. Furthermore, our rate is close to the known rate of convergence in the usual smoothing spline model. 

Abstract In this work we study statistical properties of graph-based clustering algorithms that rely on the optimization of balanced graph cuts, the main example being the optimization of Cheeger cuts. We consider proximity graphs built from data sampled from an underlying distribution supported on a generic smooth compact manifold$${\mathcal {M}}$$ $M$. In this setting, we obtain high probability convergence rates for both the Cheeger constant and the associated Cheeger cuts towards their continuum counterparts. The key technical tools are careful estimates of interpolation operators which lift empirical Cheeger cuts to the continuum, as well as continuum stability estimates for isoperimetric problems. To the best of our knowledge the quantitative estimates obtained here are the first of their kind. 

We propose GRAph Neural Diffusion with a source term (GRAND++) for graph deep learning with a limited number of labeled nodes, i.e., low-labeling rate. GRAND++ is a class of continuous-depth graph deep learning architectures whose theoretical underpinning is the diffusion process on graphs with a source term. The source term guarantees two interesting theoretical properties of GRAND++: (i) the representation of graph nodes, under the dynamics of GRAND++, will not converge to a constant vector over all nodes even as the time goes to infinity, which mitigates the over-smoothing issue of graph neural networks and enables graph learning in very deep architectures. (ii) GRAND++ can provide accurate classification even when the model is trained with a very limited number of labeled training data. We experimentally verify the above two advantages on various graph deep learning benchmark tasks, showing a significant improvement over many existing graph neural networks. 

We propose GRAph Neural Diffusion with a source term (GRAND++) for graph deep learning with a limited number of labeled nodes, i.e., low-labeling rate. GRAND++ is a class of continuous-depth graph deep learning architectures whose theoretical underpinning is the diffusion process on graphs with a source term. The source term guarantees two interesting theoretical properties of GRAND++: (i) the representation of graph nodes, under the dynamics of GRAND++, will not converge to a constant vector over all nodes even as the time goes to infinity, which mitigates the over-smoothing issue of graph neural networks and enables graph learning in very deep architectures. (ii) GRAND++ can provide accurate classification even when the model is trained with a very limited number of labeled training data. We experimentally verify the above two advantages on various graph deep learning benchmark tasks, showing a significant improvement over many existing graph neural networks.